1. Field of the Invention
This invention relates to a hardware architecture for implementing a restitution algorithm of a TV picture in the "40-millisecond" mode, within a decoder of a HD MAC (High Definition Television Multiplexing Analog Component) system for handling a high-resolution TV signal (HDTV).
2. Discussion of the Related Art
As is well known, high-resolution television (HDTV) presently constitutes one of the most technologically challenging applications in the electronic field. This is primarily attributed to the market potential of electronic equipment directly or indirectly associated with that kind of television. The market potential is so vast that it has become of primary interest to any of the industries involved in consumer electronics. Within this frame, considerable effort has been paid by the relevant industry for offering products of ever improving quality to an ever more comprehensive public.
In the HDTV field, a combination of the principal European manufacturers have proposed a standard called "High Definition Multiplexing Analog Component" (HDMAC) which provides the transmission of TV pictures with a resolution twice as high both horizontally and vertically, and in analog form. (This standard will be called "algorithm" hereinafter, without the term implying that processor programs are the only way of implementing the standard.) The HDMAC algorithm is format compatible with existing TV receivers preset for a low-resolution standard TV signal.
The algorithm provides some image pre-processing steps before transmission, so that an image to be transmitted is in a compatible format with that of low-resolution TV. Likewise, receiver sets should be capable of so processing the received image to restore it to its original format.
For a clearer appreciation of the invention, a brief description of the HD (High Definition) image format in the HDMAC system follows. The image picked up by a TV camera is formed of 1250 lines (of which only 1152 are active) and transmitted in interlaced fashion at a half-frame frequency of 50 Hz and a line sampling frequency of 54 MHz, which gives 1728 samples per line (of which only 1440 are active). On the other hand, the standard-resolution image format consists of half as many lines (625) and half as many samples per line (864); additionally, the sampling frequency is 27 MHz. Accordingly, the line period will be of 64 .mu.s.
Thus, to reduce the format of the HD image to that of standard definition, the number of samples transmitted must be reduced to one fourth of the number of the original picture samples. In other words, in the HDMAC system, the high-resolution image is subsampled by a factor of 4:1 at coding level, so that it can be transmitted within the band of the transmission channels normally used for the standard TV format. The application of that sample selection method (subsampling) also requires that the type of the image being currently handled be taken into account.
As such, images are classified according to their motion contents with respect to a preceding image, and at three different levels with which a number of subsampling grid patterns are associated. Specifically, they are called "stationary," "slow-motion" images, or "fast-motion" images.
In practice, the number of samples transmitted would remain constant between the modes, whereas the locations of the samples selected from the HD image would change. Accordingly, in that portion of the system which is to prepare the images for transmission, i.e., within the encoder, an image motion contents estimating block is provided which can decide on the subsampling mode for the current image.
It should be added that the motion estimate, and hence the subsampling decision, is performed on a block of 16.times.16 samples for the HD image. Consequently, each image will be divided into 6490 blocks, each to be coded separately (except for certain adjacency limitations which restrict the freedom of choice in the coding of clock sequences).
The three subsampling modes previously outlined are termed "20-millisecond," "40-millisecond," and "80-millisecond" mode, according to the time period required by the system to construct the subsampling grid pattern. FIG. 1 of the accompanying drawings shows the three grid patterns which correspond to the aforesaid three sampling modes.
As already mentioned, a reverse operation is to be carried out at the receiver so as to restore the missing samples on the HD image so as to obtain the image with its original resolution before it is conveyed to the display. Usually, the decoder restores the image to its full format by using median filter-based non-linear interpolation techniques.
More particularly, the missing samples are restored by interpolating a given number of nearby samples through a so-called working window. In switching from one mode to another, the basic structure of the working window does not change, but expands according to the density of the samples present on the grid pattern.
FIG. 2 shows three interpolation windows for luminance in the three different modes mentioned above. Similar considerations would apply to color samples, i.e., to chrominance.
FIG. 3 shows in greater detail the interpolation window for the 40 ms mode. The interpolation function currently used in the HDMAC system can be analyzed with reference to that FIG. For example, to interpolate coordinates e8 the following linear formula may be used: EQU e8=0.5*mf(B)-0.25*mf(A)-0.25*mf(C)+0.125*et++0.375e7+0.375e9+0.125e11.
Since three terms of the weighted sum are not available (i.e., e6, e8, e10, the second of which is the same pixel to be interpolated), such terms must be obtained from actually available samples. To obtain these values, a left-hand three-point median filter is used, as shown in the Figure at A, B, C. In essence, the three missing terms of the sum are substituted by the results of the filtering operation as carried out by three median filter: A replacing e6, B replacing e8, and C replacing e10.
A median filter on N samples will give in return a sample of "median" value from the N values subjected to filtering. For example, the median filter result of the values 10, 23, and 124 is 23, while that of the values 1, 120, and 122 is 120.
At the end of this step, all of the image odd line samples, that is, the whole odd filed of the interlaced signal, will have been restored. This is so because on the subsampling grid pattern for the "40-millisecond" mode only samples from the odd field of the HD picture are present, as can be deduced from FIG. 1.
However, the even lines must also be restored if the HD image is to be brought back to its original resolution. If the "80-millisecond" mode was considered, the grid patterns would have contained the samples from both image fields, and all the lines would have been restored at the end of the interpolation step. On the other hand, in the 20-millisecond mode, the pixels still missing would be obtained form the mean of two adjacent samples in the same line, an original sample and an interpolated one.
Thus, the 40-millisecond mode appears to be the only one that is still incomplete after the preceding interpolation step. That coding mode is, therefore, the most demanding from the computational standpoint, because it must include a further linear time interpolation step to accommodate the motion contents of the image.
The second portion of the algorithm in the 40-millisecond mode provides a time mean of samples from two successive in time odd fields restored in the first step. FIG. 4 illustrates diagramatically this kind of time interpolation.
It should be particularly noted that a simple homogeneous mean, that is, one involving samples at the same locations but in two odd field (J and L), would have a deleterious effect on the sequence of pictures that are presented on the display, since the time correlation of the two odd fields is insufficiently high. A low correlation is due to the presence of a certain motion rate in the image, resulting in limited correlation of the two time mean elements at homolog locations. It is for this reason that the motion contents should be taken into account when selecting the samples to be used for the time averaging operation.
In this respect, some control data, generated by a motion estimator that is incorporated in the encoder, is usually utilized. This data is referred to as "motion vector." The term "vector" is indicative of vectorial information that describes the direction of movement of the generic block. Shown in FIG. 4 are two components of the motion vector, which are symmetrical with respect to the two fields J and L.
The time interpolation allows the even field of the image to be satisfactorily generated in the 40-millisecond mode, as shown, thereby enabling the image to be displayed with its original resolution.
The non-linear interpolation based on the median filter, and the linear one based on the image motion estimate, operate on a pixel sequence in the raster format. As such, line memories (LMs) must be used for each time that an interpolation step is effected because the working window generally encompasses several successive lines of the image. In addition, since the two interpolations are independently performed in succession the need for an increased number of line memories becomes apparent.
A prior art embodiment of a TV signal decoder, for high-resolution receivers, was developed by Thomson TCE in 1989, and will be discussed hereinafter for later comparison to this invention.
FIG. 5 is a schematic block diagram of the architecture 50 of that decoder operated in the 40-millisecond mode for defining luminance. A first block 51 performs a non-linear filtering using a median filter whereby the signal's horizontal resolution can be doubled. This block 51 receives input samples pertaining to a special grid pattern of the 40-millisecond mode, as already set forth. Each decoder in the chain is split into portions 51 and 61, whereby it can receive and manage in parallel, two successive odd-field channels J, L as required to accommodate the motion. The working frequency is 27 MHz.
Shown diagramatically in FIG. 6 is the inner structure of one of the non-linear interpolators 51 and 61. At the input of each block, two line memories 53, 54 receive samples from three successive line of a subsampling grid 45 (FIG. 7) e.g., three samples "A" in FIG. 3. These samples are then available for a successive block 55, which performs a three-point median filtering.
A selector 56 is provided on a central leg 58 of the filtering block 51 or 61 to drive the introduction of a sampling time lag FF. This is necessary when moving from a working window 60 (FIG. 6A) centered on odd lines (a, e) to a window centered on even lines (c, g) of the odd field (J or L).
In fact, the odd lines of an odd field (a, e) on the grid pattern only contain odd samples, whereas the even lines of the odd field (e, g) contain just even samples.
With reference to FIG. 3, the three samples "A" will come in simultaneously from three respective line memories 54, whereas in the dual interpolation processing instance described below, the central sample (e5) will come in a clock time in advance of the other two and must be retarded by a factor FF 56A, as shown in FIG. 6. A signal LT is expressly provided for controlling the selector 56 so as to obtain line parity in the odd field. The selector is to switch over at the start of a new grid line, that is, every 64 .mu.s.
At the output of the median filter 55, there is a network 57 composed of sample retarding elements FF (clock period at 27 MHz, i.e., 37 ns) and adders (+) which allow a linear interpolation function to be implemented which comprises seven elements and provides a so-called non-linear interpolation working window.
In addition, between the selector 56 and the median filter block 55, there is extracted an original sample from the subsampling grid pattern to be output along with the interpolated sample.
Thus, at each clock period, two samples will be output for each new sample input: the original and the interpolated samples. In this way, the horizontal resolution is redoubled.
Two selectors, not shown for clarity because they are conventional, are connected at the two outputs and allow the samples to be addressed, in accordance with the line parity in the odd field, such that the even samples will appear at a first output 49 and the odd samples at a second output 59 (FIG. 5). This separation is necessary to maintain a working frequency of 27 MHz on the connecting buses. The above signal LT is adequate to control these two selectors as well. At this point, the image odd fields will have been fully restored, as is apparent from FIG. 7.
To complete the restitution in the even fields of the image, a time mean of the fields J and L is now needed, with the so-called motion vectors 47 also in mind. To this aim, a buffer storage structure 67 is used which comprises RAM stores 46, as shown in FIG. 5. The structure 67 has at least four connection routes between the interpolators 51, 61 and the memories 46, two for the odd line samples of the odd field and two for the even line samples. Such connection routes relate to the two fields (J and L), and each supply a structure 70 shown in FIG. 8. A shift register (SR8) 40 functions to assemble the samples into sets of eight within storage words of sixty four bits, which words are alternately written to two RAM stores 46 via a selector 44. The samples are led one at a time to the serial input of the shift register 40, and a word of sixty four bits is written to the memory every eight oncoming samples. The capacity of each memory is of 540 words, that is, five lines of 864 samples, each sample being eight bits. As a result, the memory write rate will be of 1.6875 MHz (i.e. 27/16 MHz).
Two double paris 63, 64 of shift registers are provided at the output of each memory 46 to receive two words of sixty four bits obtained from each memory by a read operation. The second register 65 of each pair 63 or 64 allows the sample to be shifted to the serial output of the other shift register. This enables the eight samples of a current word to be pipelined with those of a previously read word.
After the register pairs 63, 64, a cascade of FF sample retarding elements 66 is provided which has varying length. Each series is controlled by the horizontal component of the motion vector, thereby selecting the horizontal offset required for accommodating the motion.
The vertical component of the motion vector is used to set the memory address of the sixty four-bit word for a read operation. Thus, at each clock period, four different samples will be available at the output of each memory.
FIG. 9 enables this feature to be better appreciated by comparison to FIG. 4. It can be seen that since the even lines of J and L (lines b, d, f) are absent, a further interpolation operation is needed to determine when the value of the motion vector requires a sample from the missing field, namely a sample of the horizontal component of the motion vector, in this case equal to 2. This further interpolation is performed on four samples made available from the memory, all in the same column. Conversely, where the sample is actually present, e.g., in the instance of the vertical component of the motion vector being an odd value, it will be sufficient that the sample be sent on the four outputs to cancel the effect of the vertical interpolation weighted sum (in fact, the sum of the weights would be 1). FIG. 10 completes the structure of the decoder architecture operating in accordance with the prior art. The block 75 shown therein implements the time mean of the two fields J and L to generate an intermediate field (designated K). This structure is repeated for the even samples.
The four inputs to the block 75 (even and odd ones for J and L) are first multiplied by a selectable coefficient (0.5 or 1 for J, and 0.5 or 0 for L), and then conveyed as appropriate to an adder 69 which will output the samples.
The selection of the multiplier coefficients is carried out with field periodicity (51 Hz) by a signal FP enabling the time mean to be executed with coefficients 0.5, where an even field is to be produced, or with coefficients 1 for J and 0 for L, where an odd field is to be produced. The latter case is the same as performing no mean.
Shown in FIG. 11 are by comparison the patterns versus time of the two timing signals LT and FP used in the system.
The foregoing discussion was offered to broadly define the function of the coding and decoding algorithm in the HD-MAC system.